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NAME:  Science  of  Information,  Computation  and  Fusion 


BRIEF  DESCRIPTION  OF  PORTFOLIO: 

Information  fusion  requires  understanding  of  information  across  multiple 
domains.  Challenges:  (1)  how  to  formalize  domains  of  knowledge  and 
information  -  structures  and  their  interconnection;  (2)  how  to  mechanize 
patterns  of  reasoning  in  terms  of  computation. 


LIST  SUB-AREAS  IN  PORTFOLIO: 


•Sub-Areas 

Objectives 

Data  Discovery 

Information  Science 

•  Discover  structures  in  data  and  shape  them  into  information 

•  Methods  for  fusing  heterogeneous  data 

•  Higher-level  information  and  computation 

Algorithmic  Development 
Statistical  Modeling 

•  Exploit  structures  "1  t« 

.  r  ,  .  r-  To  improve  processing 

•  Leverage  a  priori  models  J 

Sensors  &  Radars 
Communication  Networks 

•  Fusion  of  sensors  and  fusion  of  network  data  for  detection 

•  Performance  of  sensor  fusion 
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Portfolio  (FY12) 


Universities 
LRIR 
MURI 
STTR 


itiesj_ 


Core 
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|  Program  Contribution: 

Cyber  Domain 

•  Information  Triage 

Autonomy 

•  Automated  Reasoning 

m  Ulumon-  'oohino  Intarfono  M 

..  •'**  _  ...  Space  Situational 

ligence,  Surveillance,  K  Awareness 

Reconnaissance  a  e 
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Collaborations  &  Transitions 


•  OSD/ONR  -  Data  to  Decision 

•  OSD/AFRL/RH  -  Autonomy 

•  DARPA/AFRL/RY  -  Mathematics  of  Sensing, 
Exploitation  and  Execution  (MSEE) 

•  ARO’s  MURIs: 

o  Multivariate  Heavy-tailed  Statistic 
o  Revolutionizing  High-dimensional  Microbial 
Data  Integration 

•  Topological  techniques  for  data  analytics  transitioned  to 
Army’s  Phase-ll  SBIR. 

•  Compressed-sensing  techniques  transitioned  to  Trusted 
Access  Program  Office  (TAPO). 
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Future  Direction 


Sub- Areas 

Objectives 

Data  Discovery 

Information  Science 

•  Higher-level  information  and  computation 

Algorithmic  Development 
Statistical  Modeling 

Sensors  &  Radars 
Communication  Networks 

•  Fusion  of  sensors  and  fusion  of  network  data 
for  detection 

•  Performance  of  sensor  fusion 

De- 
''"'Phasized 


Rationales: 


•  Avoidance  of  duplication 

•  Lack  of  attention  to  higher-level  information  and  computation 
grounded  on  basic  research 

•  Emerging  scientific  developments  in  this  research  area 
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Higher-level  Information  &  Computation: 
Ingredients  and  Recipes? 


Primitives 

for 

structures 


Language 

for 

specification 


Model  , 
of 

computation 


Programming 

Constructive 

Mathematics 


Cognitive 

Psychology 


Program  Challenges: 

Higher-order  Structures 
Computability 


Logic 

for 

Inference 
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Highlights  of  Research  Projects 
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FY10  MURI  -  Control  of  Information 
Collection  and  Fusion 


•  Jadbabaie 

•  Koditschek 

•  Kumar 

•  Ribeiro 
4  P  Berkeley 


Ramachandran 

•  Sastry 

•  Tomlin 

MM  Illinois 

•  Baryshnikov 
Minnesota 


•  Giannakis 

•  Roumeliotis 
Melbourne 

•  Howard 

•  Moran 


Objective:  To  formulate  a  new  perspective  on  the  joint  control  of 
heterogeneous  information  sources  to  simultaneously  achieve 
quantified  informational  and  physical  objective. 

Related  MURI  Topics  of  ARO: 

•  Opportunistic  Sensing 

•  Value  of  Information 

AFRL/RY’s  Involvement: 

•  Connection  with  the  LRIR  project  on  Layered  Sensing 

•  Interaction  between  MURI  team  and  RY’s  technical  staff 

•  Agent  for  DARPA/DSO’s  MSEE 

In  Connection  with  AFOSR  Programs: 

•  Intersection  with  all  three  sub-areas 

•  Dynamics  and  Control 

•  Sensing,  Surveillance,  and  Navigation 
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FY10  MURI  -  Control  of  Information 
Collection  and  Fusion 


RCA.l  -  Representations 
Clustering  Asymmetric  Data 
Sparsity  Control  for  Outlier  Detection 
Self -Organizing  Flocks 
Topological  Perplexity 
Hierarchically  Clustered  Formations 
Hierarchically  Clustered  Formations 


RCA.2  -  Physical-Information  State 


Control  of 
Information 


|SensorAction Plannii^ 


Partition  Based  Localization 
Hybrid  (analog/quantized)  Estimators 

LPureuit^EvasionjorSituation^ 


Information 

Sensing 


RCA.3  -  Control-Information  Linkage 


Self -Organizing  Flocks 


Topological  Perplexity 
Hierarchically  Clustered  Formations 


Hierarchically  Clustered  Formations 
Sensor  Action  Planning 
Partition  Based  Localization 
Hybrid  (analog/quantized)  Estimators 
Incentivizing  Team  Coordination 
Pursuit-Evasion  for  Situational  Awareness 


Robust  Information  Storage 
Distributed  Data  Censoring 
Sparsity-Aware  Information  Compression 
Distributed  MAP  Estimation 

|sensoi^Jnformatjo^ 

Robust  Localization 


ooperativeActive Sensm£ 


] 


]  Cooperative  Active  Sensing 
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FY10  MURI  -  Control  of  Information 
Collection  and  Fusion 


Goals: 

•  Investigate  the  (higher-order)  structures  of  these  spaces. 

•  Formulate  control  theory  on  these  spaces. 

•  Formulate  information  sensing  on  these  spaces. 


i 
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FY10  MURI  -  Control  of  Information 
Collection  and  Fusion 


Control  of 
Information 


Pursuit- 

Evasion 

Game 


Initial  Condition:  J{x,  0)  =  <Pr 

Modified  Hamilton-Jacobi-lsaacs  Equation: 

dJ(xA)  .  dJ(x,t) 

— +  min[0,  H*(xt  — ^)]  =  0 

_  Subject  to:  J(x,t)  >  -<Pa 


5 

V 

-  /fia;11 

R 

Rd 

0 

(  F  Jr 

f D 

lag  Return 


Full  Game 


Next:  Increasing  Game  Complexity 

•  Multiple  defenders  versus  attacker. 

•  More  complex,  interactive  strategies  between  the  attacker  &  defender. 

•  More  complicated  attacker-defender  configurations. 
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FY10  MURI  -  Control  of  Information 
Collection  and  Fusion 


W 


Information 

Sensing 


f{X]9) 


Model  of  Sensor 


Sensor  Information  (T  (0))-  -  =  E 


Parameterized  by  a 
configuration  space  S 


Idea:  each  sensor  configuration  gives  a  Riemannian  metric  on  S 


THE  UNIVERSITY  OF 


JSA  '  infinite  Riemannian 

manifold  if  S  is  a  manifold 


Goal:  Study  path  planning 
on  S  using  structures  on  jsA 


MELBOURNE 
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Highlights  of  Research  Projects 
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•Sub-Areas 

Objectives 

Data  Discovery 

Information  Science 

•  Discover  structures  in  data  and  shape  them  into  information 

•  Methods  for  fusing  heterogeneous  data 

•  Higher-level  information  and  computation 

Algorithmic  Development 
Statistical  Modeling 

•  Exploit  structures  L  -ix 

r  ....  r  To  improve  processing 

•  Leverage  a  priori  models  J 

Sensors  &  Radars 
Communication  Networks 

•  Fusion  of  sensors  and  fusion  of  network  data  for  detection 

•  Performance  of  sensor  fusion 

Some  Pis: 

•  J.  Culbertson  (AFRL/RY) 

•  G.  Carlsson  (Stanford  U.) 

•  C.  Priebe  (Johns  Hopkins  U.)  &  D.  Marchette  (NSWC-Dahlgren) 

•  T.  Klausutis,  S.  Butler,  J.  Curtis  (AFRL/RW) 

•  D.  Blei  (Princeton  U.) 

•  R.  Devore  (Texas  A&M  U.) 

•A.  Montanari  (Stanford  U.) 

•A.  Hero  III,  R.  Raich  (U.  Michigan  &  Oregon  State  U.) 
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w  Category  Theory  &  Layered  Sensing 

J.  Culbertson  et  al.,  AFRL/RY 


□  Project  begins  in  FY12. 

□  Layered  Sensing  aims  to  generate  situational  awareness 

from  available  resources  ( sensors ,  platforms,  networks, 
human  operators,  decision  makers)  with  required  attributes 
(persistent  coverage,  trusted  sensing,  information  triage, 
adaptability,  understanding  social  dimensions)  [ 

□  Goals: 

•  To  model  heterogeneous  systems  with  uncertainty  in  a  universal  context 
and  layers  of  abstraction. 

•  To  study  basic  properties  of  these  systems:  composability,  stochasticity, 
relations. 

•  To  develop  a  model  for  Layered  Sensing  &  use  this  model  for  fusion. 

Technical  Approach: 

Category  of  Probability  “* 
of  Spaces 


Objects  of  Study 


Probability  spaces  (ft,£fi,P)  and  maps  between  them 
9  ft  is  the  event  space 
9  Eq  is  a  (j-algebra  on  ft 
9  p  :  ->■  [0, 1]  is  a  probability  measure 
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Data  Fusion  via  Topology 

G.  Carlsson  et  al. 


Current  Tasks: 

1 .  Data  fusion:  To  study  maps  between  simplicial  complexes  (models  of 
topological  spaces. 

2.  Information  discovery:  Morse-theoretic  decomposition  of  spaces  for 
finer  analysis. 

3.  Sampling  issue:  To  study  how  different  samples  affect  the  outcomes. 


quotient  by  boundary 


optimal  chain  map 


quotient  by  boundary 


Application  of  Past  Results:  Identification  Breast-Cancer 
Patients  with  Unique  Mutational  Profile  &  Excellent  Survival 

In  Science  News  Prime,  Oct.  3,  2011  &  PNAS  2011 


£R+  sequence 
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w  Data  Fusion  via  Embedding  &  Matching 


C.  Priebe,  D.  Marchette 


New: 


Old:  Aq,  Ai  are  used  to  construct  p0,  p1 


M  = 


Aq  W 


w 


T  A 


1 


Commensurability  -  W is  used  for  matching 
Fidelity  -  A  is  used  for  preserving  dissimilarities 


Classification  of  English/French 
Wikipedia  documents 


-  jofc 

-  pom 
• • •  cca 

- r~ 

0.0  0.2  0.4  0.6  0.8  1.0 


v 
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w  Manifold  Embeddings  &  Target  Acquisition 


T.  Klausutis,  S.  Butler,  J.  W.  Curtis,  AFRL/RW 


Mission:  Autonomous  Target  Acquisition 

•  Find  and  Fix:  Target  detection  &  identification 

•  Track:  Tracking  &  aimpoint  selection 

•  Assess:  Battle  damage  indication  and  assessment 


Technical  Challenges: 

•  Multiple  platforms  coordinate  to  find  and  track  targets 

•  Computational  capabilities  vary  with  platforms 

•  Platforms  may  have  different  sensor  capabilities 


Scientific  Opportunities: 

•  Nonlinear  manifold  learning,  interactive  or  supervised  learning  for  classification. 

•  Low-dimensional  data  representation  and  analysis. 

•  Compressed  sensing  and  information  reconstruction. 
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w  Manifold  Embeddings  &  Target  Acquisition 


T.  Klausutis,  S.  Butler,  J.  W.  Curtis,  AFRL/RW 


Current  Progress: 

•  Investigating  nonlinear  dimensionality  reduction  to  reduce  computational 
overhead. 


•  Comparing  PCA  and  its  nonlinear  counterparts. 


•  Verifying  the  nonlinear  techniques  on  different  datasets. 


81 -D  feature  vectors  reduced  to  2-D 
(Ladar  Data) 

Next: 


pca  Eigenvectors  ^  ISOMAP  (k-5)  Eigenvectors 

I  •  AanlMR  I  ,  » 

|  «  CwhlWBl  '  , 


4\ _ I _ I _ I _ i _ I _ 

_ l _ i _ | _ | _ |  -4  -3  -2  -1  t  1 

m  *  m  m  Onnionl 

Separation  of  Targets:  PCA  vs.  ISOMAP 
(Synthetic  LWIR  Data) 


•  Acura  LWIR 

•  Corala  LWIR 


_ 

2 


3 

no* 


Study  sparse  and  low-dimensional  machine  learning  in  this  context. 
Study  algorithm’s  robustness  to  variations  in  scale,  intensity,  etc. 
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I Dirichlet  Process  Mixtures  of  Generalized  Linear  Models 

D.  Blei  et  al. 


P  cv/  DP[aG o), 

5cT 

II 

9y,i)\P~P> 

Xi\Qijc 

Yi 

~  GLM('\Xi,  0^y) 

mnnn 


X 


Key  Ideas: 

•  Local  grouping  of  X and  /with  varying  mixture  components. 

•  The  number  of  groups  need  not  be  known  in  advance. 

•  Xcan  simultaneously  be  categorical  and  numerical  variables. 
•A  tool  that  can  model  many  response  types  and  many  response 
shapes. 


The 

Economist 


World  politics  Business. &  finance  Economics  Science  &  technology  Culture  The  World  in  2012  Blogs  Debate  Multimedia  Print  edition 


New 

_  Regression 
Technique 


Recent  research  featured  in  The  Economist 


Organising  the  web 

The  science  of  science 

How  to  use  the  web  to  understand  the  way  ideas  evolve 

Apr  28th  201 1  |  from  the  print  edition 


Princeton 

University 


COMPUTER  scientists  have  long  tried  to  foist  order  on  the  explosion  of  data  that  is  the  internet. 
One  but  tagging  it  all  comprehensively  by  hand  is 

impossiblelDavid  Blei,  of  Princeton  University,  Ihas  therefore  been  trying  to  teach  machines  to  do 
the  job.  I  • 
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Scientific  Problem:  How  to  recover  a  function  in  high  dimension  with  small  errors 
when  only  a  finite  number  of  evaluations  are  known? 


Assumptions:  (1 )  some  form  of  sparsity 

(2)  reduced  number  of  independent  variables. 


Theorem  (DeVore-Petrova-Wojtaszczyk)  Suppose  that 

*  feCs(Q) 

»  There  is  a  g  and  j1, . . .  ,jd  (both  unknown  to  us)  such 
that  ||/(a?i, . .  .,xD)  -  g(xjl , . . . ,  xjd)\\c^  <  e. 

*  By  making  n  =  C(d ,  S)Ld  log2  D  non-adaptive  point 
queries  we  can  recover  /  by  /  to  the  accuracy 

11/  -  /llc(Q)  <  C(s,  d){\\g\\Cs{l0,1]d)L-s  +  e} 


Another  Project:  Reconstruction, 
compression  of  terrain  point  cloud  data 
using  simple  function  approximation 


Point  Cloud 


Reconstruction  using  PLANE,  77*  =  0.006 
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New  joint  work  with  D.  Donoho  and  A.  Javanmard 


imn  matrix 


y  =  Ax  +  w,  x  ~  px 


Result  -  Suppose  x  ~  px.  Sensing  matrix  A  of  this  type  can  be  constructed  such  that: 

•  If  y=Ax  is  observed,  then  xcan  be  recovered  with  high  probability  under  some 
hypotheses  on  m,  n,  and  px . 

•  The  recovery  is  robust  to  noise. 


Notes: 


There  are  instances  in  which  this  new  result  supersedes  the  current  one. 
Sparsity  of  x  is  replaced  by  x  ~  px  and  other  hypotheses  on  x. 
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w  Sensor  Fusion  via  Shannon/Renyi  Entropy 


A.  Hero  III,  R.  Raich 


Motivation:  Sensor  fusion  via  Shannon’s  information  theory  requires  computation  of 
entropy  or  divergence.  Nuisance:  Joint  probability  density  function  is  not  known. 


samples 


G</) = J 

'  g(f(x))f(x)dfi(x)  ■■ 

G(/)  - 

T£>(f(x,)). 

2=1 

IVjXjv+i,. .  •  j Xjv+M } 

Result: 

•  Derive  Variance  &  Bias 

•  Establish  Gaussian  central  limit  behavior 

•  Extend  to  more  general  cases 


osu 


UNIVERSITY  OF  (^011  State 


I 

'bq  -0.5 


MICHIGAN 


500 


- Ground  truth 

— Entropy  estimate 


1000  1500  2000 

Time  sampled  ,5sec 


2500  3000 


UNIVERSITY 
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Summary 


\ 

Current 

Portfolio 

_ J 


WARNING 

vts 

•  To  formalize  and  mechanize  information-related  concepts 
in  a  computational  framework. 

\$> 

•  Constant  need  to  develop  more  techniques  to  deal  with 
diverse  types  of  information. 

CHALLENGES 

•  Bridging  the  gap  between  humans  and  sensors  in  the 

AHEAD 

information  domain. 

I  '  i 
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